Constrained Portfolio Management Using Action Space Decomposition for Reinforcement Learning

Abstract Financial portfolio managers typically face multi-period optimization tasks such as short-selling or investing at least a particular portion of the in specific industry sector. A common approach to tackle these problems is use constrained Markov decision process (CMDP) methods, which may suffer from sample inefficiency, hyperparameter tuning, and lack guarantees for constraint violations. In this paper, we propose Action Space Decomposition Based Optimization (ADBO) optimizing more straightforward surrogate task that allows actions be mapped back original task. We examine our method on two real-world data construction tasks. The results show new consistently outperforms state-of-the-art benchmark approaches general CMDPs.